Rate Distortion for Lossy In-network Function Computation: Information Dissipation and Sequential Reverse Water-Filling

We consider the problem of distributed lossy linear function computation in a tree network. We examine two cases: (i) data aggregation (only one sink node computes) and (ii) consensus (all nodes compute the same function). By quantifying the accumulation of information loss in distributed computing, we obtain fundamental limits on network computation rate as a function of incremental distortions (and hence incremental loss of information) along the edges of the network. The above characterization, based on quantifying distortion accumulation, offers an improvement over classical cut-set type techniques which are based on overall distortions instead of incremental distortions. This quantification of information loss qualitatively resembles information dissipation in cascaded channels [1]. Surprisingly, this accumulation effect of distortion happens even at infinite blocklength. Combining this observation with an inequality on the dominance of mean-square quantities over relative-entropy quantities, we obtain outer bounds on the rate distortion function that are tighter than classical cut-set bounds by a difference which can be arbitrarily large in both data aggregation and consensus. We also obtain inner bounds on the optimal rate using random Gaussian coding, which differ from the outer bounds by O( √ D), where D is the overall distortion. The obtained inner and outer bounds can provide insights A preliminary version of this work was presented in part at the 53rd Annual Allerton Conference on Communication, Control and Computing, 2015. This work is supported in part by the National Science Foundation under grants CCF-1513936, by ECCS1343324 and CCF-1350314 (NSF CAREER) for Pulkit Grover, and by Systems on Nanoscale Information fabriCs (SONIC), one of the six SRC STARnet Centers, sponsored by MARCO and DARPA. Y. Yang, P. Grover and S. Kar are with the Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213, USA. Email: {yyaoqing,pgrover,soummyak}@andrew.cmu.edu. January 16, 2017 DRAFT ar X iv :1 60 1. 06 22 4v 2 [ cs .I T ] 1 3 Ja n 20 17